Distributed Principal Subspace Analysis for Partitioned Big Data: Algorithms, Analysis, and Implementation

Principal Subspace Analysis (PSA) -- and its sibling, Component (PCA) is one of the most popular approaches for dimensionality reduction in signal processing machine learning. But centralized PSA/PCA solutions are fast becoming irrelevant modern era big data, which number samples and/or often exceed storage computational capabilities individual machines. This has led to study distributed solutions, data partitioned across multiple machines an estimate principal subspace obtained through collaboration among It this vein that paper revisits problem under general framework arbitrarily connected network lacks a central server. The main contributions regard threefold. First, two algorithms proposed can be used PSA/PCA, with case other (raw) features. Second, sample-wise algorithm variant it analyzed, their convergence true at linear rates established. Third, extensive experiments on both synthetic real-world carried out validate usefulness algorithms. In particular, MPI-based implementation interplay between topology communications cost as well effects straggler